feat: add TORCHX_IMAGE to env vars for Docker-based schedulers (#1128) #1129
Conversation
|
For your consideration, @kiukchung @andywag @d4l3k |
meta-cla
bot
added
the
CLA Signed
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1129 +/- ##
=======================================
Coverage 91.63% 91.63%
=======================================
Files 83 83
Lines 6392 6397 +5
=======================================
+ Hits 5857 5862 +5
Misses 535 535
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
The lint failure seems unrelated, @kiukchung |
this is unrelated to your changes I'll override it. We're experiencing a few broken assets due to the migration of this repo from github.com/pytorch/torchx to github.com/meta-pytorch/torchx fix for doctest is: #1131 but it seems that our AWS creds are broken at the moment. |
|
Sounds good, @kiukchung! Looks like |
|
@kiukchung has imported this pull request. If you are a Meta employee, you can view this in D83282650. |
6 participants
Adding
TORCHX_IMAGEto environment variables so that we give a chance to the payload to record the value beyond the retention restrictions of the scheduler so as to facilitate reproducibility.Test plan:
[x] updated unit tests
[x]
local_docker --workspace ""-> no image built or pushed -> we expect to see the base image inTORCHX_IMAGEenv varstorchx run -s local_docker --workspace "" utils.sh --image alpine:latest -- env[x]
local_docker-> image built locally, but not pushed -> we expect to see local image SHA inTORCHX_IMAGEenv varstorchx run -s local_docker utils.sh --image alpine:latest -- env[x]
aws_batch --image_repo-> image built locally, tagged and pushed to image repo -> we expect to see image tag inTORCHX_IMAGEenv varstorchx run -s aws_batch --scheduler_args 'queue=<queue-name>,priority=<priority>,image_repo=<image-repo>,share_id=<share-id>' utils.sh -h g5.4xlarge --image alpine:latestObserve in Batch Job details: